\(L^p\) convergence
\(L^p\) convergence
Definition 1 (Convergence in \(L^p\)). Let \(p \ge 1\), then we say that a random sequence \(X: \Omega \to \R^\N\) defined on a probability space \((\Omega,\sF,P)\) converges in \(L^p\) to a random variable \(X_\infty:\Omega \to \R\), if The convergence in \(L^p\) is denoted by \(\lim_nX_n = X_\infty\) in \(L^p\).
Remark 1. For \(p \in [1, \infty)\), the convergence of a random sequence \(X: \Omega\to\R^\N\) in \(L^p\) to a random variable \(X_\infty:\Omega \to \R\) is equivalent to
Proposition 2 (Convergences \(L^p\) implies in probability). Consider \(p \in [1, \infty)\) and a sequence of random variables \(X:\Omega\to\R^\N\) defined on a probability space \((\Omega,\sF, P)\) such that \(\lim_nX_n = X_\infty\) in \(L^p\), then \(\lim_nX_n = X_\infty\) in probability.
Proof. Proof. Let \(\epsilon > 0\), then from the Markov’s inequality applied to random variable \(\abs{X_n-X}^p\), we have ◻
Example 3 (Convergence almost surely doesn’t imply convergence in \(L^p\)). Consider the probability space \(([0,1], \sB([0,1]), \lambda)\) such that \(\lambda([a,b]) = b-a\) for all \(0 \le a \le b \le 1\). We define the scaled indicator random variable \(X_n: \Omega \to \set{0,1}\) such that We define \(N = \set{0}\), and for any \(\omega \notin N\), we can find \(m \triangleq \lceil \frac{1}{\omega}\rceil\), such that for all \(n > m\), we have \(X_n(\omega) = 0\). Since \(\lambda(N) = 0\), it implies that \(\lim_nX_n = 0\) a.s. However, we see that \(\E\abs{X_n}^p = \frac{2^{np}}{n}\).
Remark 2. Convergence almost surely implies convergence in probability. Therefore, above example also serves as a counterexample to the fact that convergence in probability doesn’t imply convergence in \(L^p\).
Theorem 4 (\(L^2\) weak law of large numbers). Consider a sequence of uncorrelated random variables \(X:\Omega\to\R^\N\) defined on a probability space \((\Omega,\sF, P)\) such that \(\E X_n = \mu\) and \(\Var(X_n) = \sigma^2\) for all \(n \in \N\). Defining the sum \(S_n\triangleq \sum_{i=1}^nX_i\) and the \(n\)-empirical mean \(\bar{X}_n \triangleq \frac{S_n}{n}\), we have \(\lim_n\bar{X}_n=\mu\) in \(L^2\) and in probability.
Proof. Proof. From the uncorrelatedness of random sequence \(X\), and linearity of expectation, we get It follows that \(\lim_n\bar{X}_n = \mu\) in \(L^2\). Since the convergence in \(L^p\) implies convergence in probability, the result holds. ◻
Theorem 5 (\(L^1\) weak law of large numbers). Consider an random sequence \(X:\Omega\to\R^\N\) defined on a probability space \((\Omega,\sF, P)\) such that \(\E\abs{X_1}< \infty\) and \(\E X_1 = \mu\). Defining the sum \(S_n\triangleq \sum_{i=1}^nX_i\) and the \(n\)-empirical mean \(\bar{X}_n \triangleq \frac{S_n}{n}\), we have \(\lim_n\bar{X}_n=\mu\) in probability.
Example 6 (Convergence in \(L^p\) doesn’t imply almost surely). Consider the probability space \(([0,1], \sB([0,1]), \lambda)\) such that \(\lambda([a,b]) = b-a\) for all \(0 \le a \le b \le 1\). For each \(k \in \N\), we consider the sequence \(S_k = \sum_{i=1}^ki\), and define integer intervals \(I_k \triangleq \set{S_{k-1}+1, \dots, S_{k}}\). Clearly, the intervals \((I_k:k \in \N)\) partition the natural numbers, and each \(n \in \N\) lies in some \(I_{k_n}\), such that \(n = S_{k+n-1}+i_n\) for \(i_n \in [k_n]\). Therefore, for each \(n \in \N\), we define indicator random variable \(X_n: \Omega \to \set{0,1}\) such that For any \(\omega \in [0,1]\), we have \(X_n(\omega) = 1\) for infinitely many values since there exist infinitely many \((i,k)\) pairs such that \(\frac{(i-1)}{k} \le \omega \le \frac{i}{k}\), and hence \(\lim\sup_nX_n(\omega) = 1\) and hence \(\lim_nX_n(\omega) \neq 0\). However, \(\lim_nX_n(\omega) = 0\) in \(L^p\), since
\(L^1\) convergence theorems
Theorem 7 (Monotone Convergence Theorem). Consider a non-decreasing non-negative random sequence \(X: \Omega\to\R_+^\N\) defined on a probability space \((\Omega,\sF,P)\), such that \(X_n \in L^1\) for all \(n \in \N\). Let \(X_\infty(\omega) \triangleq \sup_nX_n(\omega)\) for all \(\omega \in \Omega\), then \(\E X_\infty = \sup_n\E X_n.\)
Proof. Proof. From the monotonicity of sequence \(X\) and the monotonicity of expectation, we have \(\sup_n\E X_n \le \E X_\infty\). Let \(\alpha \in (0,1)\) and \(Y: \Omega\to\R_+\) a non-negative simple random variable such that \(Y \le X_\infty\). We define From the monotonicity of sequence \(X\), the sequence of events \(E \in \sF^\N\) are monotonically non-decreasing such that \(\cup_{n \in \N}E_n = \Omega\). It follows that We will use the fact that \(\lim_n\E[Y\Ind{E_n}] = \E[Y]\), then \(\alpha\E Y \le \sup_n\E X_n\). Taking supremum over all \(\alpha \in (0,1)\) and all simple functions \(Y \le X_\infty\), we get \(\E X_\infty \le \sup_n\E X_n\). ◻
Theorem 8 (Fatou’s Lemma). Consider a non-negative random sequence \(X: \Omega\to\R_+^\N\) defined on a probability space \((\Omega,\sF,P)\). Let \(X_\infty(\omega) \triangleq \lim\inf_nX_n(\omega)\) for all \(\omega\in\Omega\), then \(\E X_\infty \le \lim\inf_n\E X_n.\)
Proof. Proof. We define \(Y_n \triangleq \inf_{k \ge n}X_k\) for all \(n \in \N\). It follows that \(Y: \Omega \to \R_+^\N\) is a non-negative non-decreasing sequence of random variables, and \(X_\infty = \sup_nY_n = \lim_nY_n\). Applying monotone convergence theorem to random sequence \(Y\), we get \(\E X_\infty = \sup_n\E Y_n\). The result follows from the monotonicity of expectation, and the fact that \(Y_n \le X_k\) for all \(k \ge n\), to get \(\E Y_n \le \inf_{k\ge n}\E X_k\). ◻
Theorem 9 (Dominated Convergence Theorem). Let \(X: \Omega \to \R^\N\) be a random sequence defined on a probability space \((\Omega, \sF, P)\). If \(\lim_nX_n = X_\infty\) a.s. and there exists a \(Y: \Omega \to \R_+\) such that \(Y \in L^1\) and \(\abs{X_n} \le Y\) a.s. for all \(n\in\N\), then \(\E X_\infty = \lim_n\E X_n\).
Proof. Proof. From the hypothesis, we have \(Y+X_n \ge 0\) a.s. and \(Y-X_n \ge 0\) a.s. Therefore, from Fatou’s Lemma and linearity of expectation, we have
2 &Y + X__n (Y+X_n) = Y + _nX_n,& &Y - X__n (Y-X_n) = Y - _nX_n.
Therefore, we have \(\limsup_n\E X_n \le \E X_\infty \le \liminf_n\E X_n\), and the result follows. ◻
Convergence theorems for conditional means
Proposition 10. Let \(X:\Omega\to\R^\N\) be a random sequence on the probability space \((\Omega, \sF, P)\) such that \(\E\abs{X_n} < \infty\) for all \(n \in \N\). Let \(\sG\) and \(\sH\) be event spaces such that \(\sG, \sH \subset \sF\). Then the following theorems hold.
Conditional monotone convergence theorem: Consider a random sequence \(X:\Omega\to\R_+^\N\) that is non-negative and non-decreasing a.s.. We define \(X_\infty:\Omega\to\R_+\) for each \(\omega \in \Omega\) as \(X_\infty(\omega)\triangleq \sup_{n \in \N}X_n(\omega)\). If \(X_\infty \in L^1\), then \(\E[X_n\mid \sG] \uparrow \E[X_\infty\mid\sG]\) a.s.
Conditional Fatou’s lemma: Consider a random sequence \(X:\Omega\to\R_+^\N\) that is non-negative and non-decreasing a.s.. If \(\lim\inf_n X_n \in L^1\), then \(\E[\lim\inf_nX_n\mid \sG] \le \lim\inf_n\E[X_n\mid \sG]\) a.s.
Conditional dominated convergence theorem: Consider a random sequence \(X:\Omega\to\R_+^\N\) and a random variable \(Y:\Omega\to\R_+\) such that \(Y \in L^1\) and \(\abs{X_n} \le Y\) a.s. for all \(n \in \N\). If \(\lim_n X_n = X_\infty\) a.s., then \(\E[X_n\mid \sG] \to \E[X_\infty\mid\sG]\) a.s. and in \(L^1\).
Proof. Proof. Let \(X:\Omega\to\R^\N\) be a random sequence on the probability space \((\Omega, \sF, P)\) such that \(X_n \in L^1\) for all \(n \in \N\).
Conditional monotone-convergence theorem: By monotonicity, we have \(\E[X_n\mid \sG] \uparrow Y\) a.s. where \(Y:\Omega\to \R_+\) is \(\sG\) measurable. The monotone convergence theorem implies that, for each \(G \in \sG\),
Conditional Fatou’s lemma: Defining \(Y_n \triangleq \inf_{k\ge n} X_k\), we get \(Y_n \uparrow Y_\infty = \lim\inf_k X_k\). By monotonicity, The conditional monotone-convergence theorem implies that
Conditional dominated-convergence theorem: By the conditional Fatou’s lemma, we have and the a.s.-statement follows.
◻